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Abstract 

The purpose of this paper is to compute asymptotically Hankel determinants for 
weights that are supported in a semi-infinite interval. The main idea is to reduce the 
problem to determinants of other operators whose determinant asymptotics are well 
known. 



1 Introduction 

The main purpose of this paper is to compute asymptotically the determinants of the finite 
matrices H n (u) defined by 

det(ai + j)™j =0 



where 



rod 

a i+ j = / x l+ ^u{x) dx 
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with suitable conditions on the weight function u(x). These determinant entries depend only 
the sum i + j and are hence classified as Hankel determinants. 

Hankel determinants such as these were considered by Szego in [TO] and also by Hirshmann 



in H, but in both cases for finite intervals. These determinants are important in random 
matrix theory and its applications. 

Our main result is as follows. Suppose we replace u{x) by a function given in the form 
w(x)U(x) where w(x) is the weight e~ x x u with v > —1/2. Then if U is nowhere zero and 
U—l is a Schwartz function (a condition which can be considerably relaxed) the determinants 
are given asymptotically as n — > oo by 

det (H n (u)) = expjcin 2 logn + c 2 n 2 + c 3 n\ogn + c 4 n + ^n 1 ^ 2 + c 6 logn + c 7 + o(l)} (1) 

where 

ci = 1, c 2 = -3/2, c 3 = v, 

2 f°° 2 
C4 = —v + log27r, C5 = — / logfLux ))da; 

7T JO 

c 6 = z/ 2 /2 - 1/6, c 7 = 4/3 log G(l/2) + (1/3 + i//2) log tt + (z//2 - 1/18) log 2 

- log Gil + v) - u/2 log U(0) + / xS(x) 2 dx, 

2n 2 Jo 

G is the Barnes G-function, and S(x) = J Q °° cos(xy) \ogU {y 2 )dy . 

The idea behind the proof is to replace the matrix H n (u) with one whose i, j th entry is 
given by 

roo 

Pi(x)Pj(x)u(x) dx 



where the Pj's are orthogonal (Laguerre) polynomials with respect to the weight e"V '. 

These new determinants can then be evaluated using the ideas of the "linear statistic" 
method in random matrix theory. More precisely, the above determinant can be replaced by 

det (7 + K n ) 

where K n is an integral operator whose kernel is given by 

71-1 

J2LKx)L-(y)(u-i)(y) 

with L\ defined as the i th Laguerre function of order v. The main computation in the paper is 
to approximate the above kernel with a different kernel which involves Bessel functions. This 
new kernel was fortunately already considered in . There, asymptotics for certain integral 
operators were computed and these results are then applied to give the result of formula ([!]). 
The kernels considered in arises in random matrix theory in the "hard-edge" scaling for 
ensembles of positive Hermitian matrices. Details about the random matrix connections can 
be found in [0. 

The formula in (1) was stated earlier in 0] where a heuristic argument using the Coulomb 
gas approach was used to derive the same formula. The Coulomb gas approach was also used 
in P] to extend to the case where the interval of integration is the entire real line 
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2 Preliminaries 



We first show how to replace the powers in the Hankel determinants with the orthogonal 
polynomials. Let 



'Hn(u) = ( / Pi(x)Pj(x)u(x) dx 



n—l 



i,j=0 



and write 

Pi( x ) = Yj aikxk - 



k<i 



Lemma 2.1 Let Pi,7i n (u), H n (u) be defined as above and let us assume that all moment 
integrals exist. Then 

det (H n (u)) = A* det (#„(«)), 

where 



n—l 



^ = (n 



O; 



\2 



i=0 



Proof. We have 

roc 

det(H n (u)) = det( / a jm x m )(^a k ix l )u(x) dx). 

m<j l<k 

The j, k entry of the matrix is given by 

POO 

J2 a i™ a ki / x m+l u(x) dx 

m<j l<k J ° 
roc 

= J^a-jm / x m+l u(x) dx a H . 

m<j l<k 

Form this it follows that TC n (u) is a product of three matrices TH n {u)T t . The matrix T is 
lower triangular with entries a jm , m < j. It is easy to see from this that the lemma follows. 
The next step is to evaluate the term 

n—l 

A n = ( dii) , 

which is of course straight forward. We normalize the polynomials so that they are orthonor- 
mal. Then it is well known |J that 

1 



2 _ 

aii ~ T(l + i + u)T(l + i) 



The following lemma computes the product asymptotically using the Barnes G-function. 
This function is defined by [I2f 

oo 

G{l + z) = (2n) z / 2 e^ z+1 >^ Ez2 / 2 l[( K (l + z/k) k e- z+z2/2k ) (2) 



k=l 



with 7^ being Euler's constant. 
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Lemma 2.2 

n-1 



k 1 = n 



i=0 

zs green asymptotically by 

exp jciin 2 logn + d 2 n 2 + logn + <i 4 n + c? 5 logn + d e + o(l)| 

C?l = 1, C?2 = — 3/2, C?3 = Z/, C?4 = — Z/ + log 27T, c/5 = z/ 2 /2 — 1/6 

4 = 4/31ogG(l/2) + (1/3 + z//2)logvr + (i//2 - 1/18) log 2 - logG(l + v) 

Proof. It is well known that the Barnes G function satisfies the property |l~2[ 

G(l + z) =T(z)G(z). 

From this it is quite easy to see that A' 1 can be written as 

G(l + n)G(l + n + v) 
G(l + u) ' 



The asymptotics of the Barnes function are computed in |L2| and since G(l + a + n) is 
asymptotic to 

n (n+a) 2 /2-l/12 e -3/4n 2 -an^2 7r )(n+«)/2 G 2/3^ 1/ /2) 7 r 1 / 6 2 _1/36 

we can directly apply this formula with a = and a = v to obtain the desired result. 

This last result shows then that the asymptotics of the Hankel matrices can be reduced 
to those of the matrices Ti, n (u). We compute these asymptotics by replacing the determinant 
with a Fredholm determinant. To do this we must first consider some estimates for the 
various kernels. 



3 Hilbert-Schmidt and trace norm estimates 

We proceed to write the determinant of 7i. n (u) as a Fredholm determinant. It is clear that if 
U(x) is a bounded function then the matrix 7i n (u) can be realized as P n M\jP n where My is 
multiplication by U and P n is the projection onto the space spanned by the first n Laguerre 
functions. This is clearly a bounded, finite rank operator defined on L 2 (0, oo). Thus 

det H n (u) = det(I + K n ) 

is defined where K n is the integral operator whose kernel is given by 

Y,Lt{x)my){U{y)-l). 

i=0 
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Let us write a square root of the function U — 1 as V. Then 



det(I + K n ) = det(J + P n M v M v P n ) = det(J + M v P n M v ). 

This last equality uses the general fact that 

det(J + AB) = det(J + BA) 

for general operators defined on a Hilbert space. At this point we have replaced the kernel 
K n with 

n— 1 

J2V(x)L»(x)L»(y)V(y). 

i=0 

Our primary goal in the paper is to replace this last kernel, which we will commonly refer to 
as the Laguerre kernel, with a more familiar one namely the compressed Bessel kernel, also 
defined on L 2 (0, oo), 

Y^ J^jnxy^^yj'Mny)^) - J u {2{ny)^) j'Mnx)^) 

x-y 

We use the term "compressed" since the above without the factors V(x) and V(y) is generally 
called the Bessel kernel. 

We will use the general fact that if we have families of trace class operators A n and 
B n (thinking of the Laguerre kernel as A n and the compressed Bessel kernel as B n ) that 
depend on a parameter n such that the Hilbert Schmidt norm \\A n — B n \\2 = o(l) and 
tr (A n - B n )(I + B^- 1 = o(l) then 

det(7 + An)/ det(I + B n ) -»• 1 

as n — > oo. The proof of this fact uses the idea of generalized determinants found in 0, and 
also requires uniformity for the norms of the inverses of the operators I + B n . This will be 
made more precise later. 

Using the standard identity for the sum of the products of Laguerre functions allows us 
to write the Laguerre kernel 

n-l 



£ V(x)mx)L»(y)V(y) 



i=0 

as 



n(n + u)V(x) V(y). 

x-y 

Notice that the above kernel has somewhat the same form as the compressed Bessel kernel. 
It turns out that both the kernel above and the Bessel kernel have an integral representation 
that make the Hilbert-Schmidt computations simpler. The next lemma shows how this is 
done for in the Laguerre case. 
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Lemma 3.1 We have 

L "- l(x)L " ( ^lf"- l(y)L " (3:) = \ [\LUWK(ty) +LU(ty)K(tx))dt. 

JL> (J £ J 

Proof. Call the left-hand side <&(x,y) and the right-hand side ^>(x,y). It follows from the 
differentiation formulas for Laguerre polynomials that there is a differentiation formula 



d_ 

dx 



( L 



V 

n-l 



X 



V Ufa) 



( A 




where 



1 v 
-x 

2 2 

An easy computation using this shows that y) satisfies 



A(x) = -x---N, B(x) = C(x) = ^/N(N + v) 



On the other hand, if we temporarily assume that v > and differentiate under the integral 
sign and then integrate by parts we find that 

{ x lL + y ^ + 1 ) VJ/(x ' y) = ^ L n-i( x ) L n(y) + K-i{y)K{x)\- 

Hence 

( d d \ 

h— +y— + l\ ($(x, y) - y)) = 0. 

From this it follows that &(x, y) — ty(x,y) is of the form ip(x/y) e~( x+y " 2 for some function 
cp of one variable. Still assuming v > 0, both y) and ^(x, y) tend to as i and y tend 
to independently. This shows that if = and so the identity is established when v > 0. 
Since both sides of the identity are analytic functions of v for Re v > — 1 the identity holds 
generally. 

Thus our kernel may be rewritten 

y n (n + u)V(x)V(y) [[L^tx) L»(ty) + L^ty^tx)) dt. (4) 

We assume from now on that \v\ > 1/2. The compressed Bessel kernel appearing in (j^) also 
has the following well-known integral representation 



nV{x)V(y) / J u (2Vmci)J u (2y/riyi)dt (5) 



and the rest of the section is devoted to replacing the Laguerre functions by the appropriate 
Bessel functions in (|]) and showing that this leads to a small Hilbert-Schmidt error and then 
showing that certain traces tend to zero. Notice that in the above notation kernel (4) is the 
same as A n and kernel (5) is the same as B n . We need only the following fact about the 
Laguerre functions. 
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Lemma 3.2 Let a be any real positive constant and suppose that v > —1/2. Then asn — > oo, 
the normalized Laguerre functions satisfy 

max x l/4 L v n (x) = O^ 1 / 4 ), 

0<x<a 

maxima;) = 0(n~ 1/4 ). 
VFe remark that the implied constants in the estimates may depend on a but not on n. 
Proof. These estimates follow easily from formula (7.6.9) and Theorem 8.91.2 in ||. 

Lemma 3.3 We have 

U n [x) - c n v J v (2^f~Nx) = ^N~^ 4 x 5/4 sin(2v / iV^ - a) + 0{N- b ' 4 (x~ l/4 + x 3 )), (6) 

6V7T 

where N = n + [y + l)/2,c n>1/ = A r-^( r ^+ 1 ) )i/2 ; a = („/ 2 + 1/4)^ fln d ^ e constant 
implied in the O depends only on v. 

Proof. Formula (8.64.3) of Szego reads, in current notation, 

VJx) - c n ^J v (2^[Nx) 

[J v {2y/Nx)J- v {2y/Ni) - J_ i ,(2v / iVx) J u (2y/Nt)] t U n {t) dt. 
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4 sin z/7T jo 

Using the Lemma |3.2| estimate for L^{t) we find that if Nx < 1 the right side is at most a 
constant times 

(Nx)-^ /2 [ X (Nt)-^ /2 t I/Jt) dt = OiN-M-^xi-M) = 0(N- 2 ). 
Jo 

(If v is an integer this must be multiplied by logiV.) 

So we assume Nx > 1. The integral over Nt < 1 is at most a constant times 

(Nx)- 1 / 4 ( l,N (Nt)-W % N- X IH* dt = 0(N-'ix- 1 ' 4 ) 
Jo 

with, possibly, an extra logiV factor. So we confine our integral now to Nt > 1. 

If we replace the Bessel functions by their first-order asymptotics the error is at most a 
constant times 

N- 1 / V 1/2 + t-y 2 )(xt)-^N-y A {t + t 3/4 ) dt = 0(N~ 5 ^(1 + x)). 



Therefore with this error we can replace the Bessel functions by their first-order asymptotics, 
obtaining, after using some some trigonometric identities, 

^N~ l / 2 x~ 114 sin(2v / iV^ - 2\fWt) t 3/4 L u n (t) dt. (7) 



Notice that this has the uniform estimate 0(iV~ 3 / 4 (l + s 3 / 2 )), and the earlier errors 
combined are 0(iV~ 5//4 (a;~ 1//4 + x). It follows that if in this integral we replace L^(t) by 
c n ,uJv{2\/Ni) the error is 0(iV~ 5 / 4 (l + x 3 )). Then if we replace J I/ (2\/ Nt) by its first-order 
asymptotics the error is at most a constant times 

N -i/2 x -i/* f x ^(Nt)-^dt = 0(N-^% 

Jl/N 

Hence with the sum of the last-mentioned errors we may replace the factor L^(t) in (|7|) by 
the first-order asymptotics of c n:U J u {2\fNt). Using a trigonometric identity shows that this 
results in 



8v^ 



N -a/4 x -i/4 f x r sin (2 v / iV^ - a) - sin(2v / iV^ - A^Nt + a)] t 1/2 dt 

Jl/N 



!^ n -3/a x 5/4 sin ( 2v /jVx _ a ) + (N' 



9/4 x -l/4N 



+ 0(N' 



-5/4 x 3/4N 



Putting these things together gives the statement of the lemma. 

In what follows we use the notation o%{ ■ ) or Oi( ■ ) for a family of operators whose Hilbert- 
Schmidt norm satisfies the corresponding o or O estimate, or for a kernel whose associated 
operator satisfies the estimate. Similarly, o\ ( • ) and 0\ ( • ) refer to the trace norm. We also 
set N' =n+(i/-l)/2. 



Lemma 3.4 . Suppose V is in L°° and satisfies \V(x)\ 2 (x 1 ^ 2 + x 6 )dx < oo. Then the 
difference between the kernel (fjj and 

\^n{n + v)V{x)V{y)c n ^ v c n , v f\j v {2VWti)J v {2^NUj) + J v {2^my)J v {2VNt^)} dt 



is equal to 

(i) 0\(N~ l l 2 ) plus a constant which is 0(1) times 



(txf /A sm(2VN'tx - a)J u (2^Nty) + {tyf /A sin(2y / A% - a)J u (2V 'N'tx) 



V(x)V(y) dt 



plus a similar term with x and y interchanged; 
(u)0 2 (N~ 1 ^). 

Proof. The estimate of Lemma 3.3 holds with n replaced by n — 1 if N is replaced by N'. 
Write 



L v n _ x {x)L v n {y) - c n _^c n ,M2VN^)J u (2^Ny) 
[Ll_ x (x) - c n ^ u J u (2VN^)] L v n {y) + L v n _ x {x)\Ll{y) - c n , v J v {2yfNx~)] 
+ [c n .^J u (2VN^) - Ll^WL^y) - c n „J v (2^fm)\. 
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The contribution of the right side to the difference of the two kernels is a constant which is 
0(1) times 

Nj* {[Ll_Sx)-c n ^ v J v {2VN^)]L v n {ty) 

+L u n _ 1 {tx)[L u n {ty) - c n ^J u {2VNtx)] 
+ [cn^J^VWte) - /£.i(te)K(ty) - Cn^Jvpy/Nte)]) V(x)V{y) dt. 

The integrand is a sum of three terms, each of which is the kernel (in the x, y variables) 
of a rank one operator. The trace norm of such an integral is at most the integral of the 
Hilbert- Schmidt norms.) Using this fact and Lemma 3.2 we see that the contribution of 
the error term in (|6]) to any of these terms is Oi(iV -1 / 2 ), as is the contribution of the last 
term above. Then we see that replacing the two Laguerre functions by the corresponding 
Bessel functions leads to an even smaller error. The error term in Lemma 3.3 is seen also 
to contribute Oi{N~ 1 / 2 ). Applying this lemma, and then doing everything with x and y 
interchanged we arrive at the statement of part (i). 

For part (ii), we have to show that the integrals are 02(iV~ 1//2 ). It is easy to see that 
with error Oi(iV -1 / 2 ) we may replace the Bessel functions by their first-order asymptotics, 
resulting in a constant which is 0{N^ 1 ^ A ) times 

V(x) V(y)x 5/4 y- 1/4 ^ sm(2\fWtx - a) cos(2^Nty -a)tdt 

plus similar expressions. A trigonometric identity and integration by parts shows that the 
integral is at most a constant times 

max (1, |2v / iV^-2y / iV^|))- 1 + max (1, \2yfWx~ + 2^JNy~\ - 2a))" 1 , 

and an easy exercise shows that, together with the outer factors, this gives (^(iV -1 / 4 ). 
Analogous argument applies to the other integrals and this completes the proof of the lemma. 



Lemma 3.5 The kernel (TJj is equal to 

nV(x)V(y) f 1 J u (2Vn~te)J u (2^Ej)dt (8) 
J o 

plus an error 02(1). 

Proof. Since the Laguerre kernel has Hilbert-Schmidt norm n 1//2 (the operator is a rank n 
projection) multiplying (|j) by constants which are 1 + 0(n~ 1 ) produces an error C^^ -1 / 2 ). 
The constants we choose are n/(^Jn(n + v)c n _i tU c n!U ). It follows from this and Lemma 3.4 
that with error 02(1) we can replace (|J) by 



V{x)V{y) j\j u (2Vl^)J u (2^Nty) + J^^ty) J,(2v / iVte)] dt. (9) 
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Now we show that if we replace N and N' by n in this kernel the error is 02(1). Let's 
look at the error incurred in the integral involving the first summand when we replace N' 
by n. It equals 

^-^=y \ZtxJ' u (2y/rtx)J v (2^Nty)dt. (10) 
If ntx < 1 and nty > 1 the inner integral is at most a constant times 

/■min(l, 1/nx) 
n - M /2-l/2 x - M /2 / 'r^ 2 dt, 

Jo 

where \x = max(— u, 0) as before. This is bounded by a constant times 

n~ l/2 (nxy il/2 if nx < 1, n~ 3/2 x~ 1 if nx > 1. 

This times V(x)V(y) is 02{n~ 1 ) and so its eventual contribution to the Hilbert-Schmidt 
norm (because of the external factor n and the fact that r ~ n) is O^n^ 1 ^ 2 ). 
If ntx < 1 and nty < 1 the inner integral is at most a constant times 

, /o /o /o /" min ( 1 ' V™^. 1M) 

n - M _i/2 a ,- M /2 y - M /2 y 

By symmetry we may assume y < x. If nx < 1 this is at most a constant times the outer 
factor which, when multiplied by V(x)V(y), is 02(n~ 3 / 2 ). If nx > 1 the above is at most a 
constant times 

^-3/2+^/2^-1+^/2 -/i/2 

and this times V A (a;)V A (?/) is C^n -1 ). Thus the eventual contribution of the portion of the 
integral where ntx < 1 is 02(n~ 1 ^ 2 ). 

In the region ntx > 1, if we replace the first Bessel function in ([!(]) by the first term of 
its asymptotic expansion it is easy to see we incur in the end an error 2 {n- 1 ' i ). After this 
replacement the inner integral becomes a constant times 



n 



1/4 x 1/4 f 1 t 1/4 cos(2 v / ^ : - a) J v {2JNty) dt 

Jl/nx 



f t 1/4 cos(2Jrtx/n - a) J v (2y/ty) dt. 

Jl/x 

Taking account of the external factor of n in our kernel, and the r-integral, we see that we 
want to show that the Hilbert-Schmidt norm of the kernel 

V(x)V(y) n~ l [ n t 1/4 cos(2 Jrtx/n - a) JJ2^/ty) dt 

Jl/x V 

tends to as n — > 00. But from the asymptotics of the Bessel function it is clear that 
n^ 1 times the integral is uniformly bounded by a constant times 1 + y~^l 2 and tends to 
whenever x 7^ y. Hence the dominated convergence theorem tells us that the product is 
o 2 (l). 
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An analogous argument shows that replacing N' by n in the second summand of @ and 
then replacing N by n in both summands leads to an error 02(1). This completes the proof. 

Remark. In the preceding lemmas our various kernels had the factor V(x)V(y). It is 
easy to see from their proofs that the lemmas hold with this factor replaced everywhere by 
Vi(x)V 2 (?/) as long as V\ and V 2 are bounded and have sufficiently rapid decay at infinity. 
The next lemma is the first that will require some smoothness. 

We shall denote by J n (x,y) the kernel (|5|) without the external factor V(x)V(y), and by 
J n the corresponding operator. As before we denote by My multiplication by V so that (|) 
is the kernel of the operator My J n My. 

Lemma 3.6 For any Schwartz function W the commutator [W, J n ] has Hilbert- Schmidt 
norm which is bounded as n — > 00. 

Proof. Write the kernel of the commutator as in formula (3). It has the form 

W(x) - W(y) , 
x-y 

plus a similar term with x an y interchanged. If nx > 1 and ny > 1 then the product 

J v (2 y/nx) y/ny J v (2 y/ny) 

is 0(x _1//4 ?/ 1//4 ), and thus if \x — y\ is bounded away from zero, or if we integrate over any 
bounded region the Hilbert-Schmidt norm (the square root of the integral of the square) is 
bounded. If \y — x\ < 1, and assuming x,y > 1 we see that y/x is bounded and thus the 
Hilbert-Schmidt norm can be estimated by the square root of 

J\ h x — y 

But this is known [[□]] to be bounded by /f^ |x|iy(x)| 2 (ia;. Now suppose that nx < 1 and 
ny > 1. Then the product of Bessel functions is 0(r?T M+1 / 4 ar' 1 / 2 ?/ 1 / 4 ). If \x — y\ is bounded 
away from zero, then the resulting Hilbert-Schmidt norm is 0(n _1//4 ). If \x — y\ < 1, then it 
is also clear that the Hilbert-Schmidt norm is 0{n~ 1 ^). The other two cases xn > 1, yn < 1 
and xn < 1 , yn < 1 are handled in the same fashion and are left to the reader. 

Lemma 3.7 For any bounded functions W\ and W2 we have 

tr (A n - B n ) W 2 JnW^O 

as n — > 00. 
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Proof. For convenience all kernels K(x, y) in this proof will be replaced by their unitary 
equivalents 2^/xyKix 2 , y 2 ). We denote by L n (x,y) this unitary equivalent of the Laguerre 
kernel ([|) without the external V factors and by J n (x,y) here the unitary equivalent of the 
Bessel kernel (|]) without the V factors. We also make the substitution t — > t 2 in the t 
integrals. Thus in our present notation 



J n (x, y) = AnJxy \ J u {2\fnxt) J u (2\fnyt) tdt. 
Jo 



(12) 



We also denote by J n (x, y) the unitary equivalent of the first displayed operator of Lemma 3.4, 
without the V factors. Thus 



J n (x, y) = 2Jn(n + v)c n ^ u c n)h 



x J^y \ JJ(2VN'tx)JJ2VNty) + J U (2V N'ty)J u {2^ Ntx)]tdt. (13) 
jo 

If we set Vi(x) = V(x 2 )Wi(x 2 ) then we see that our trace equals tr M Vl (L n — J n )M V2 J n . 
We shall show that this goes to in two steps, showing first that tr My 1 (L n — J n )My 2 J n — > 
and then that tr My^Jn — Jn)My 2 — > 0. 

First, the asymptotics of the Bessel functions gives 



Ju{z) 




2 ( z' 1 ' 2 ' 
cos(;z — a) + O \ 

TTZ \ < Z > 



where < z >= (1 + z 2 ) 1 ^ 2 . (This also uses v > —1/2.) Hence 



J n (x,y) = -n^fxy 

7T JO 



cos(2 v / ntx — a) 
(2 v ^tx) 1 /2 



+ 



( (y/ntx 



,-1/2 



y< (\/ntx) > 



X 



cos(2y/nty — a) 



+ 



y/nty) 



-1/2 



tdt 



(2^ityy/ 2 \< {y/nty) > 

8 r l 

—n I cos(2\/ntx — a) cos(2\rnty — a)dt 
IX Jo 



-0 



n 



n 



n 



\< \/nx > < \/nx > < \/nx > x / 2 < y/ny > 1 / 2 y 

The last summand is at most a constant times the sum of the preceding two. Using this, a 
trigonometric identity and integrating we find that 



J n {x,y) = O 



n 



+ 



n 



+ 



+ 



n 



< \fn[x — y) > < y/n(x + y) > < y/nx > < y/ny > 



(14) 



We consider first My 1 (L n — J n )My 2 J n . Lemma 3.4 tells us that the kernel of My 1 (L n — 
J n )My 2 is Oi(n~ 1//2 ) plus the unitary equivalent of the expression in part (i) with modified 
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V factors. (See the remark following Lemma 3.5.) In the proof of part (ii) it was stated 
that if we replace the Bessel functions in this expression by their first order asymptotics the 
error is 0\(n~ l l 2 ). (We shall go through the details for similar integrals below.) So we may 
replace L n — J n by a constant which is 0(1) times 

/ [x 3 sin(2V^V 7 ta; - a) cos{2y/Nty - a) + y 3 sm(2y/N>tya) cos(2v / iVtx - a)] t 3 dt. 
Jo 

(Recall the unitary equivalents we are using and the variable change t — > t 2 .) If in the 
integrals we made the replacements N, N' — > n we would incur an error 0((x 4 + y 4 )/ y/n). 
Multiplying this by Vi(x)V2(y) times (fUD and integrating is easily seen to give o(l). After 
these replacements the integral becomes what may be written 

(a; 3 - y 3 ) [ sin(2y/ntx - a) cos(2y/nty - a) t 3 dt + y 3 [ sm(2y/nt(x + y) — 2a) t 3 dt 
Jo Jo 

= o( x3 - y3 + x3 -y 3 , y 3 \ 



< y/n(x — y) > < y/n(x + y) > < ^/n(x + y) > 
O 



x 3 — y 3 x 3 + y 



< y^n(x — y) > < y^n(x + y) > J 

Let us see why if we multiply this by Vi(x)V2(y) times (0) and integrate we get o(l) 
First, 

x 3 — y 3 y/n ( x 2 + y 2 \ 



< y/n(x — y > < y/n(x — y) > \< y/n~(x — y) > J 

goes to zero pointwise and this times Vi(x)V2(y) is bounded by a fixed L 1 function. Thus the 
integral of the product goes to zero. The term with < \/n(x-\-y) > instead of < y/n(x — y) > 
is even smaller. 
Next consider 

x 3 — y 3 y/n 



< y/n(x — y) > < y/nx > 

We may ignore the factor x 3 — y 3 since it may be incorporated into the V^. After the 
substitutions x — ► x/y/n, y — > y/ y/n the integral in question becomes 



1 = f r \Vx(x/y/n-)V2(y/Vh~)\ 
n J J <x — y > < x > 



Schwarz's inequality shows that the y integral is 0(n 1//4 ) so our double integral is bounded 
by a constant times 



n-v r dx + r \y^m dx = + n -m r \}m dx = i og 

Jo Jl X A/Vn X 

Again the term with < y/n(x + y) > instead of < y/h~(x — y) > is even smaller. 
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Now we look at My x {J n — J n )My 2 J n . To find bounds for J n (x,y) — J n (x,y) let us look 
first at the error incurred if in the first integral in fll3|) we replace N' by n. The error in the 
integral together with the external factor Jxy equals 



dr 



A" 



r jo 



x 



3/2 y 1/2 J' u (2^tx)J u (2^ty) t 2 dt. 



Using the asymptotics of J v {z) and the fact that 




— sin(2; — a) + 0(z 



-3/2i 



we can write the above clS db constant times 



n dr 



A" 



r Jo 



x 3/2 y l/2 



sin(2y / rtx — a) 



( v ^te) 1 /2 



+ 0{{y/nt. 



X 



-3/2^ 



X 



cos(2y/Nty - a) 



O 



( (V^/)~ 1/2 



t 2 dt. 



'Nty) 1 / 2 " V< (V^tj/) >, 

We estimate the trace norm of Vi(x)V2(y) times this by taking the trace norm under the 
integral signs. Since the integrand is, for fixed r and t, a function of x times a function of 
y its trace norm equals the product of the L 2 norms of its factors. In multiplying out we 
will have main terms and error terms and we must estimate norms of all products. Thus we 
compute (in each line there will be an integral corresponding to a main term and then and 
error term) 



x 3 |Vi(x)p 
y/rtx 



dx = ofa-Wr 1 ), 



x 3 \Vi(x)\' 
(y/ntx) 3 



dx = o(n~ 3/2 r 3 ), 



y\V 2 {yW 



d y = o(n- x/2 r x ), 



y(VNty)- l \V 2 {y)[ 



dy = o(rr l r 2 ). 



'Nty " " J <VNty> 2 

Combining L 2 norms we see that the trace norm of the contribution to the integrand of 
all but the product of the main terms is 0(n~ 3 ^t~ 2 ). Integrating over t we are left with 
0(n -3 / 4 ) and integrating over r gives 0(n _5//4 ). If we combine this we the external factor 
in fllBp which is 0(n) we are left with 0(n~ l l A ). Since the operator norms of the J n are 
bounded the eventual contribution to the trace of the product will be 0(n -1 / 4 ). 
Thus we are left with the main term, which is 

rn rl 

/ (rN)~ 1/4 — xsm(2y/rtx - a) cos(2VNty - a)tdt. 
J at 7 Jr JO 



If in this we replaced r and N by n everywhere in the integrand the error would be 0(n 3 ^ 2 ). 
If we multiply this by Vi(x)V2(y) and use the estimate (fH]) we find by dominated convergence 
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that the product has trace tending to zero, even keeping in mind the extra factor 0(n) in 
(|13|). So we may make these replacements, which results in 

(n — N') rT 1 \ x sm(2y/ntx — a) cos(2y/nty — a) tdt. 
Jo 

Now there is a second integral in J n , which is obtained from the first by interchanging x 
and y. Interchanging and adding gives what can be written 



in 



N') n 1 \(x — y) sm(2^ s /ntx — a) cos{2\fnty — a) + y sm(2\fntx + 2\fnty — 2a)] tdt 
Jo 

n~ 3 / 2 (x — y) n~ 3 / 2 (x + y) \ 



O 



< y / ri(x — y) > < ^/n(x + y > 



If we multiply by Vi(x)V 2 (y) and use the estimate flT4]) we find again that the product has 
trace tending to zero, even keeping in mind the extra factor 0(n). 

Thus replacement of N' by n in ( Jl3| ) leads to an eventual error in the trace of o(l). 
Similarly so does then the replacement of N by n. Finally, 

yjn(n + v)c n -i, v Cn >u = n (1 + O^ 1 )), 

so the eventual error in the trace upon replacing the constant by n is 0(n~ l ) times what 
is obtained by multiplying Vi(x)V2(x) by the square of ( |HD and integrating. Dominated 
convergence shows this also to be o(l). This completes the proof of the lemma. 



4 Completion of the proof 

Recall that A n is the Laguerre kernel (H) and B n is the compressed Bessel kernel (^), which 
we also denote in its operator version as My J n M v . We shall show first that 

det (I + An) ~ det (J + B n ) 

as n — > oo. (It is clear that A n is a finite rank operator and so it is trace class, and using 
the integral representation for the compressed Bessel kernel and integrating over t shows 
that B n is also trace class. Thus both determinants are defined.) This will follow from 
what we have already done once we know that the operators I + B n are uniformly invertible, 
which means that they are invertible for sufficiently large n and the operator norms of their 
inverses are 0(1). 

Lemma 4.1 The operators I + B n are uniformly invertible and 

(I + B n y l =1- M v J n M vu -i + O a (l). (15) 
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Proof. We replace the operator by its unitary equivalent M v J n M v where now J n is given 
by (|12"D, or equivalently 

JJx, y) = Jxy \ JJxt) Jy{yt) tdt 
Jo 

and we set V(x) = V(x 2 ). If we set H(x, y) = ■JxyJ v {xy) with H the corresponding operator 
(the Hankel transform), and denote now by P n multiplication by the characteristic function 
of (0, 2-y/n), then J n = HP n H and so I + B„ is unitarily equivalent to I + M v HP n HM v . 
These operators will be uniformly invertible if I + P n HM v2 HP n are. 

Now it has recently been shown Q that the operator HM v2 H is of the form jy(V 2 ) +K, 
where ^(V 2 ) denotes the Wiener-Hopf operator with symbol V(x) 2 = V^a; 2 ) 2 and K is a 
compact operator on L 2 (R + ). (Much less is needed for this than that V 2 be a Schwartz 
function.) Since 1 + V(x 2 ) 2 = U(x 2 ) is nonzero and, being even, has zero winding number 
it follows from general facts about truncations of Wiener-Hopf operators that the operators 
I + P n W(V 2 )P n are uniformly invertible. Then since K is compact it follows that the 
/ + P n (W(V 2 ) + K)P n will be uniformly invertible if the limiting operator / + W(V 2 ) + K = 
I + HMy 2 H is invertible. (For an exposition of the facts we used here see, for example, 
Chap. 2 of 0.) However, since H 2 = I the inverse of I + H M y2 H is easily seen to be 
I + H W H where W = —(1 + V 2 )" 1 ^ 2 . This establishes the first statement of the lemma. 

For the second statement we apply Lemma 3.6 and use the facts J 2 = J n (which follows 
from H 2 = I) and that V 2 is a Schwartz function to see that for any bounded function W 
we have 

(J + B n ){I + MyJ n M W ) = (I + M V J n My){I + MyJ n M W ) 

= 1 + MyJ n M v+V 2 W+w + 2 (1) = / + M v J n M v+uw + 2 (1). 
If we choose W = —VU~ X and multiply both sides by (I + -B n ) _1 we obtain the result. 



Lemma 4.2 det (J + A n ) ~ det (J + B n ) as n — > oo. 

Proof. If an operator C is trace class then 

det (I + C) = det (J + C)e~ tTC 

where det is the generalized determinant [[?]. (The generalized determinant is defined for 
any Hilbert-Schmidt operator.) Hence we can write 

= det (/ + (A, - B„)(I + B„)- v ) = det (/ + (A, - B„)(I + B,,y l ) e -tr(A,-a.)(/+a.)-'. 

It follows from Lemmas 3.4(h) and 3.5 that t4„ — B„ — > in Hilbert-Schmidt norm, and 
therefore from the uniform invertibility of the I + B n that the same is true of (A n — B n )(I + 
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Bn) -1 . Therefore from the continuity of the generalized determinant in Hilbert-Schmidt 
norm that we conclude that the generalized determinant above has limit 1. Thus it suffices 
to show that tr (A n — B n )(I + B n )~ l — > as n — > oo. Since A n — B n — > in Hilbert-Schmidt 
the 02(1) term in (pT5|) contributes o(l) to the trace of the product. By Lemma 3.7 the term 
My J n M vu -i in fljjD also contributes o(l). That tr (A n — B n ) itself is o(l) follows easily from 
arguments already given — one can check that at each stage the traces of the error operators 
tend to zero. 

Finally, we can quote the main result of [0 which gives the asymptotics of det (J + B n ) 
or, more exactly the determinants of their unitary equivalents. The formula is 

9r)V2 (-00 I /-co 

det(J + B n ) ~ exp{ / log(t/(x 2 )dx - z//21ogf/(0) + — / xS{x) 2 dx] 

7r jo 2n 2 Jo 

where S(x) = J Q °° cos(xy) \og(U(y 2 )dy. This gives 

Theorem 4.3 Suppose U is nowhere zero and U — l is a Schwartz function. Then (fj[) holds. 

As mentioned in the introduction this result was computed heuristically in using the 
Coulomb fluid approach. In the same paper the analogous result was also obtained for 
weights supported on the entire real line. These determinants involve Hermite polynomials. 
It is highly likely that the results here (and techniques) could also be extended to that case. 
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